Will You Get It Right Next Week: Predict Delayed Performance in Enhanced ITS Mastery Cycle

Researchers of Intelligent Tutoring Systems (ITS) and Educational Data Mining (EDM) have focused increasing attention on predicting students’ long-term retention performance as well as attempting to find effective methods to help improve student knowledge retention. Wang and Beck proposed a system which allows ITS to strive for student long-term mastery learning. This paper describes our implemented work of such a system for improving student retention along with a model to predict student performance for delayed retention tests; this incorporates features of student behavior and performance levels in the system. Using this model, we analyzed the data of 27,451 mathematical problems that 662 students in the 2012 fall semester attempted to solve or were successful in solving. We found that after students successfully master the skill, the number of those who attempted solving problems during the process of achieving mastery is predictive of delayed retention test performance. Specifically, on the 7-day retention test, 82% of students who try to master a skill in 3 or 4 attempts did so correctly, while students who required 5 to 8 attempts to master a skill achieved a rate of 70%. Furthermore, we propose that using the prediction model to guide the improvement of our tutorial decision-making on when we should test students also help them to better retain skills.